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Purpose 


Context: Flight Test 
Supports four needs 

■ Managements approval to proceed 

■ Establish a record of accountability 

■ Inject expert insight into the risk management 
process 

■ Mitigate the human factors vulnerabilities associated 
with small group teaming and project ownership 



Purpose 


Essential is a clear statement of the boards domain of 
assessment and the roles and rules governing its 
conduct 

Complicated by a multi-organization test team 

Hypically constrained to: 

■ Airworthiness of the test vehicle/item 

Safety /Mission Assurance of the vehicle and test operations 
based on 

■ Environment 

■ Team composition 



Structure 


Prerequisites /Timing 

■ Technical reviews completed including instrumentation 

■ Close enough to flight test to 

■ Define flight test maneuvers and test approach 

■ Stabilize the tie sign by ground test results 

■ Sufficient time to respond to board actions 

Membership 

■ Organizational 

■ Multi -organizational boards complicate roles 

■ Functional 

■ Facilities, modification, instrumentation „ management 

■ Technical 

■ Fes t dis c i p 1 i nc s, te t h nt > log) - exper ts„ cngi n ce ri ng 

Involvement/ Decision Authority 



Structure 


Involvement/Decision Authority 

b Decision authority apportioned to level of perceived 
risk 

■ High complexity tests/ systems 

■ Require more time to assess 
a May require a team of specialists 

■ Independence 

a Distanced from day-to-day opera tions/teaming 
a Non-stakeholder 



Airworthines s 


Standard should be defined 

■ Depends on vehicle test outcome 

■ Research: 

■ Experimental class for commercial 

■ Gov’t agency could use internal standard 

* Military development 

■ Commercial or internal 

■ Commercial 

■ FAR Part Standards 



Airworthines s 


i Review addresses standards compliance 

i i Primary focus is safety implications 

■ Environmental suitability 

■ Safe energy transfer/ containment 

■ Flight Control 

■ Software development standards, V&V 

■ static and dynamic stability margins 

■ Adequate performance & structural margins 

■ EMI /EMC, etc... 

1 1 Ground tests are the typical evidence 



Risk Assessment 


Decision Authority 

■ Usually prescribes the methodology 

■ Complicated by multi-organizational roles - build in overlap 

Risk Characterization 

■ Hazards identified via SSWG 

■ Cause Mechanism - Consequence 

■ Mitigations should target Causes 

■ Corrective Actions address “realized” risk 

■. Consequence + probability of occurrence — risk 

■ Matrixed cross plot divided into regions of risk gradation 

. High / medium / low 



Risk Assessment 


Key risk areas with safety & mission assurance 
implications 

■ Design/Technical 

■ System (integrated) 

■ Subsystem 

m Operational 

■ Test maneuvers 

■ Environmental conditions 



Risk Assessment 


Board Guidelines 

aa Understand source of consequence and probability 

■ Data behind probability quantification./ experiential 
assessment for qualification 

■ Non-mitigated risk assessment (where did we start) 

■ Worst case consequence vs. all significant outcomes 

■ 'Total program vs. singular event exposure assessed 

■ Assumptions that underly the analysis 

■ Review non- credible hazards 

■ Proper consideration of multiple event failures 



Risk Assessment 


Board Guidelines 

■ Type of mitigations support final risk claim 

■ Over utilization of human dependent mitigations 

■ Recognition time and speed of response appropriate 
a Adequate lines of defense 

■ Training necessary to be effective 

a Communications complexity (layers, SA, language, 
seniority, personality) 

■ Integrated team response - appropriate CRM 

■ Predetermined vs. ad hoc judgment application 



Risk Assessment 

Board Guidelines 

■ Transition from many individual risks to the integrated risk 
picture 

■ Widely ignored in an qualitative assessment 

■ Must assess dependency across hazards unless willing to live with 
independent assumption (form of worst case analysis) 

■ Relies heavily on ability to stop, move to a sate condition and land 

Test Conditions really required? 

■ Traceable to a requirement 

■ Operationally suitable 



Risk Assessment 


Board Guidelines 

■ Do hazards address group HF failures 

■ Decline of independence over time 

■ Turnover of personnel 

m Increase in complacency with success 

■ “Cost of doing business” mentality 



Accountability 


■ Accountability within the process 

■ linsure sufficient time for reviews — sufficient time 
for prep material review 

■ Reasonable durations — all day? 

i Target complex areas with specialty teams and 
increased interface 

■ Capture and address all concerns 

■ Do not allow time to force a decision 



Accountability 


Accountability to the process 

■ Provide written expectations 

■ Content 

■ Timing — NBT X days before first flight 

■ Readiness 

■ Assess effectiveness 

■ Process to decision effectiveness /efficiency 

■ Decision effectiveness (long term) 

■ Poor risk characterization 
b Unidentified risk realization (process escapes) 

■ Modify and codify the process 



Accountability 


Decision Accountability 

j 

■ Clear written authorization 

■ Periodic reports 

■ End point delineated 

■ Clear closure actions with record of 

■ Clear return criteria — could be to a different 
decisional authority 



Job Done? 


No!! 

Visit — a matter of good communication not 
trust! 

Assess 

■ Culture 

m Pace of operations /Situational Awareness 

■ Understanding 



